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FIELD OF THE INVENTION 

The present invention relates to input devices 
for computers generally. 



BACKGROUND OF THE INVENTION 

U.S. Patent 4,787,051 to Olson describes an 
inertial mouse system which uses three pairs of acceler- 
ometers. 

U.S. Patent 4,839,836 to LaBiche et. al. de- 
scribes an apparatus for providing spatial orientation 
data signals, using an inertial platform accelerometer 
cluster having a plurality of accelerometers. 

U.S. Patent 5,181,181 to Glynn describes a 
computer apparatus input device, for three-dimensional 
information. 
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SUMMARY OF THE INVENTION 

The present invention seeks to provide an 
improved input device. 

There is thus provided in accordance with a 
preferred embodiment of the present invention information 
input apparatus including body supported apparatus for 
sensing voluntary body motions and providing an output 
indication thereof, a symbol output interpreter operative 
to utilize the output indication for providing symbol 
outputs, and a motion output interpreter operative to 
utilize the output indication for providing motion con- 
trol outputs. 

Further in accordance with a preferred embodi- 
ment of the present invention, the output indication 
represents features of body motion including features 
which are characteristic of the individual. 

Still further in accordance with a preferred 
embodiment of the present invention, a mode selector is 
provided which is operative to cause a selected one of 
the symbol output interpreter and the motion output 
interpreter to function. 

Further in accordance with a preferred embodi- 
ment of the present invention, the body supported appara- 
tus is a hand held device. 

Still further in accordance with a preferred 
embodiment of the present invention, the body supported 
apparatus is a generally pen-shaped device. 

Additionally in accordance with a preferred 
embodiment of the present invention, the generally pen- 
shaped device is operative to provide a visible writing 
function. 

Still further in accordance with a preferred 
embodiment of the present invention, the information 
input apparatus also includes an object whose motion is 
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controlled by the motion control outputs. 

Further in accordance with a preferred embodi- 
ment of the present invention, the object is a graphic 
object displayed on a display or a physical object. 

Further in accordance with a preferred embodi- 
ment of the present invention, the symbol outputs repre- 
sent alphanumeric symbols or a sensory quality such as an 
acoustic stimulus, including but not limited to music, or 
such as a visual stimulus, including but not limited to a 
color or a color image. 

It is appreciated that the applicability of the 
present invention is very broad and is suitable for the 
following fields of use, inter alia: games such as video 
games, toys, model vehicles, robotics, simulations such 
as flight simulations, technical drawings and CAD. 

Still further in accordance with a preferred 
embodiment of the present invention, the information 
input apparatus also includes a computer, having a loca- 
tion input and a symbol input, and a display operated by 
the computer and wherein the symbol outputs represent 
information to be displayed on the display and the motion 
outputs are supplied to the location input and are em- 
ployed by the computer to govern the location of the 
information on the display. 

Further in accordance with a preferred embodi- 
ment of the present invention, the symbol outputs include 
function commands. 

There is also provided, in accordance with a 
preferred embodiment of the present invention, a method 
by which a manipulable device provides an output indica- 
tion representing its own angular motion, the method 
including recording actual acceleration data from a 
plurality of acceleroraeters mounted in the manipulable 
device, generating predicted acceleration data on the 
basis of hypothetical angular motion information, compar- 
ing the predicted acceleration data to the actual accel- 



Patent 




W0 95/21436 PCT/US95/01483 



4 

eration data, computing improved hypothetical angular 
motion information, repeating, while the predicted accel- 
eration data differs significantly from the actual accel- 
eration data, the generating, comparing and computing 
steps, and providing an output indication of the improved 
hypothetical angular motion information. 

Further in accordance with a preferred embodi- 
ment of the present invention, the angular motion infor- 
mation includes angular displacement information, angular 
velocity information and angular acceleration informa- 
tion. 

Still further in accordance with a preferred 
embodiment of the present invention, the method includes 
computing linear motion information from the improved 
hypothetical angular motion information and from the 
actual acceleration data. 

Additionally in accordance with a preferred 
embodiment of the present invention, recording includes 
recording from at least four accelerometers mounted in 
the manipulable device, wherein the accelerometers ' each 
have a center of mass and wherein the centers of mass do 
not lie within a single plane.. 

Still further in accordance with a preferred 
embodiment of the present invention, the method also 
includes receiving the output indication of the improved 
hypothetical angular motion information and manipulating 
an object in accordance therewith. 

There is additionally provided, in accordance 
with a preferred embodiment of the present invention, an 
accelerometer array mounted in a manipulable device and 
including at least four accelerometers each having a 
center of mass, wherein the centers of mass do not lie 
within a single plane, and a manipulable device motion 
computer receiving input from the accelerometers and 
generating an output signal indicative of the motion of 
the manipulable device. 
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Further in accordance with a preferred embodi- 
ment of the present invention, the manipulable device 
motion computer is operative to: 

record actual acceleration data from the accel- 

erometers, 

s generate predicted acceleration data on the 

basis of hypothetical angular motion information, 

compare the predicted acceleration data to the 
actual acceleration data, 

compute improved hypothetical angular motion 
information, 

repeat, while the predicted acceleration data 
differs significantly from the actual acceleration data, 
the generating, comparing and computing steps, and 

provide an output indication of the improved 
hypothetical angular motion information. 

Further in accordance with a preferred embodi- 
ment of the present invention, the apparatus also includ- 
ing an object manipulator receiving the output signal 
indicative of the motion of the manipulable device' and 
manipulating an object in accordance therewith. 

There is also provided, in accordance with a 
preferred embodiment of the present invention, an infor- 
mation input method including seizing voluntary body 
motions and providing an output indication thereof, 
utilizing the output indication for providing symbol 
outputs, and utilizing the output indication for provid- 
ing motion control outputs. 



Patent provii 




WO 95/21436 



PCT/USS>5/01483 



6 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and 
appreciated from the following detailed description, 
taken in conjunction with the drawings in which: 

Fig- 1 is a simplified pictorial illustration 
of object control and handwriting recognition apparatus 
constructed and operative in accordance with a preferred 
embodiment of the present invention; 

Figs. 2A and 2B are schematic drawings of 
preferred structures of portions of the apparatus of Fig. 
1; 

Fig. 3 is a simplified block diagram of the 
apparatus of Fig. 1; 

Fig. 4 is a simplified flow chart illustrating 
the object control process performed by the apparatus of 
Fig. 1; 

Fig. 5A is a simplified flow chart illustrating 
the process of step 440 of figure 4; 

Fig. 5B comprises mathematical equations illus- 
trating the process performed by Fig. 5A; 

Fig. 6 is a simplified block diagram of the 
apparatus of Fig. 1; 

Fig. 7A is a simplified flow chart illustrating 
the teaching process performed by the apparatus of Fig. 
1; 

Fig. 7B is a simplified flow chart illustrating 
the recognition process performed by the apparatus of 
Fig. l; and 

Figs. 8A and 8B are graphical illustrations 
useful in understanding a preferred method for a portion 
of the teaching and recognition processes performed by 
the apparatus of Fig. l. 

Appendix A is a computer listing of a preferred 
software implementation of a portion of steps 720 of Fig. 
7A and 800 of Fig. 7B. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Reference is now made to Fig. l which is a 
simplified pictorial illustration of apparatus operative 
to perform motion control in synergistic combination with 
symbol interpretation such as handwriting recognition. A 
hand-held pen 10 is operative to be translated and rotat- 
ed about some or all of three perpendicular axes. The 
term "six degrees of freedom" is used herein to designate 
translation along and rotation about three orthogonal 
axes. 

Pen 10 also comprises a plurality of built-in 
accelerometers 25, such as model ICS 3031-2 commercially 
available from IC Sensors, 1701 McCarthy Blvd., Milpitas, 
CA 95035. Preferably, pen 10 comprises. six accelerome- 
ters arranged in pairs, with each pair lying along a 
particular axis, with the axes being mutually orthogonal. 
Alternatively, the axes may not be mutually orthogonal. 
In any case the accelerometers need not be coplanar. 

Pen 10 also comprises a plurality of amplifiers 
30, associated with the plurality of accelerometers 25. 
Fig. 2A is a schematic drawing of a preferred embodiment 
of amplifier 30. 

Alternatively, removable apparatus comprising a 
plurality of accelerometers 25 as described above, and 
also comprising associated amplifiers 30, as described 
above, may be retrofitted onto the pen 10. The removable 
app;:-atus may have the form of a cap fitting the end of 
the pen, a ring fitting over the pen, or any other suit- 
able form. 

In a still further alternative, the apparatus 
may not include a pen, but may have any other suitable 
hand held form. In a yet further alternative, the appa- 
ratus may be in the form of a ring fitting the user's 
finger, may be supported by the body of a user, or mount- 
ed thereupon in any suitable matter. 
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Pen 10 also comprises a switch 35, which can be 
used to send a signal indicating whether pen 10 is being 
used for handwriting recognition or as a pointing and 
control device. Alternatively the signal may be sent by 
moving pen 10 in a predefined format, or by any other 
appropriate means. During handwriting recognition, the 
user may write with pen 10 on writing surface 37. 

The data from the plurality of accelerometers 
25 in pen 10 is termed herein "accelerometer data". The 
accelerometer data is sent through a cable to a control 
circuit 40. Alternatively, the accelerometer data may be 
sent through any suitable wireless communication link, 
such as ultrasonic, infrared, or by any other suitable 
means . 

• Control circuit 40 amplifies the acceleration 
signals from pen 10 and converts them to digital form, 
preferably using an analog to digital converter. Pig. 2B 
is a schematic drawing of a preferred embodiment of an 
analog to digital converter suitable for the present 
application. 

Control circuit 40 then sends acceleration data 
to a CPU 50. CPU 50 may be any suitable CPU such as an 
IBM PC compatible computer with an 80386 processor chip. 

Associated with CPU 50 are a screen 60 and a 
keyboard 70. An object 80, such as a cursor or a graphic 
representation of a physical object, is displayed on 
screen 60. When pen 10 is used for object control, CPU 
50, based on the acceleration data, moves the cursor or 
graphic representation 80 with six degrees of freedom, 
corresponding to the movement of pen 10. 

A symbol 85, such as one or more characters or 
words, may also be displayed on screen 60. When pen 10 
is used for handwriting recognition, CPU 50, based on the 
acceleration data, displays the symbols corresponding to 
what is written on writing surface 37 on screen 60. 

The functionality of the apparatus of Fig. l 
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will now be described. Using switch 35 the user indi- 
cates whether handwriting recognition or object control 
is to be performed. Depending on the user's choice, the 
apparatus of Fig. 1 performs the appropriate function. 

The functionality of the apparatus of Fig. l 
when performing objer; v. control will now be briefly de- 
scribed. The user moves pen 10 in three dimensions; the 
motion may include six degrees of freedom. Pen 10 sends 
acceleration data describing the accelerations of pen 10 
during the motion to control circuit 40. 

Control circuit 40 amplifies and digitizes the 
acceleration data. The data is sent by control box 40 to 
CPU 50. 

CPU 50 computes the translational displacement, 
velocity, and acceleration of pen 10 along three mutually 
perpendicular axes which axes need have no relation to 
the axes of the accelerometers. CPU 50 also computes the 
angular displacement (rotation), velocity and accelera- 
tion of pen 10 around the same three mutually perpendic- 
ular axes. 

Based on the computed output, CPU 50 moves the 
cursor or the representation of an object 80 on screen 60 
with translations and rotations corresponding to those of 
pen 10. The axes for the translation and rotation of the 
cursor or object correspond to the axes used to compute 
the translation and rotation of pen 10. 

Reference is now additionally made to Fig. 3 
which is a simplified block diagram of the apparatus of 
Fig. 1. Pen 10, when moved by the user with six degrees 
of freedom, transmits data describing the accelerations 
of pen 10 to amplification circuit 120. Amplification 
circuit 120 amplifies the acceleration data and transmits 
the amplified acceleration data to analog/digital con- 
verter 130. Analog/digital converter 130 digitizes the 
acceleration data and transmits the digitized data to 
displacement/velocity/acceleration computation apparatus 
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140, termed herein DVA 140. 

DVA 140 computes the angular displacement, 
velocity, and acceleration of pen 10 around three mutual- 
ly perpendicular axes which axes need have no relation to 
the axes of the accelerometers . DVA 140 also computes 
the translational displacement, velocity and acceleration 
of pen 10 along the same three mutually perpendicular 
axes. 

DVA 140 transmits data describing the six 
degrees of freedom to screen display control ISO. Based 
on the data, screen display control 150 updates screen 60 
to show the new location and orientation of the cursor or 
the other object depicted on screen 60. 

Reference is now additionally made to Fig. 4 
which is a simplified flow chart illustrating operation 
of the apparatus of Fig. 1 in accordance with a preferred 
embodiment of the invention. The preferred method of 
operation of the method of Fig. 4 includes the following 
steps: 

STEP 410: Read accelerometer data. Data from 
each of the plurality of accelerometers 25 is sampled, 
preferably at a rate of one thousand data points per 
second. 

. STEP 412: Check whether session is at the 
beginning. At the beginning of a session, STEP 420, 
described below, is required. 

STEP 415: Check whether pen is in motion. The 
accelerometer data is analyzed to determine whether pen 
10 is in motion. 

Preferably, pen 10 is considered to be not in 
motion whenever all of the acceleration signals indicate 
that the only sensed accelerations are due to gravity. 
Signals are chosen from one member of each of three pairs 
of accelerometers, each pair arranged along a different 
axis. 

Let the vector U=(U 1< U 2 ,U 3 ) denote the signals 
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of the three acceleronieters . Let the matrix A=(K 1/ k 2/ K 3 ) 
denote the sensitivities of each of the three acceleronie- 
ters . 

The sensitivities of the accelerometers correct 
for any deviations of the axes of each pair of accelerom- 
eters as they are actually mounted in pen 10 from the 
common axis on which they are supposed to be situated; 
this component of the sensitivity is called static sensi- 
tivity. The sensitivities also correct for deviations 
between the axes of the global orthogonal coordinate 
system and the axes of the pairs of accelerometers; this 
component of the sensitivity is called dynamic sensitivi- 
ty. In actual practice, both static sensitivity and 
dynamic . sensitivity may make important contributions to 
sensitivity. 

The static sensitivity is computed as part of 
step 420, described in detail below. The dynamic sensi- 
tivity is computed as part of step 455, described in 
detail below. 

Let e denote a small positive constant; for 
example, .005g where g denotes the acceleration of gravi- 
ty at the earth's surface. Then the pen is considered 
not to be in motion whenever 1-6 < [A^U] < i+£, 

STEP 420: Compute initial conditions. The 
initial conditions may comprise the initial Euler angles 
between the global coordinate system and the axes of the 
pairs of accelerometers. These Euler angles are now 
determined. 

• Assuming that, at the initial condition, either 
at the beginning of the session or when the pen is not in 
motion, the only accelerations measured are due to gravi- 
ty, the static sensitivity can be computed from the 
accelerometer data while the pen is at rest in three 
known orientations. Alternatively, the static sensitivi- 
ty can be computed once as a property of the pen and 
stored for future use. 
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STEP 430: Compute the differential signal from 
each pair of accelerometers . The signals from each 
member of each pair of accelerometers are subtracted to 
form a differential signal for each pair of accelerome- 
ters. 

STEP 440: Compute rotational parameters. The 
rotational parameters define parameters of the motion 
about the three axes of the global coordinate system. 
The rotational parameters comprise the three Euler 
angles; the three angular velocities; and the three 
angular accelerations. 

The rotational parameters are computed in 
parallel using an iterative feedback ■ loop. In each 
iteration of the loop, an estimated differential acceler- 
ation is computed from the current rotational parameters. 
If the difference between the estimated differential 
acceleration and the actual differential acceleration 
signal is less than a predetermined amount, iteration is 
terminated. 

Otherwise, new values of the parameters are 
estimated in each iteration from the previous values and 
the difference between the estimated differential accel- 
eration and the actual differential acceleration data. 
The method of this step is described more fully below 
with reference to Fig. 5. 

STEP 450: Compute translation acceleration. 
The angular orientation and the angular acceleration are 
known from step 440. From the angular orientation and 
the sensitivity vector the acceleration due to gravity is 
computed. 

Given the angular acceleration and the acceler- 
ation due to gravity as well as the sensitivity vector, 
the translational acceleration is computed. The computa- 
tion is according to the formula a t =K" 1 u-ag-a r where a t 
is the translational acceleration; K is the sensitivity 
vector; u is the signal of one accelerometer; a a is the 
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component of the acceleration of gravity sensed by the 
one accelerometer; and a r is the angular acceleration. 

STEP 455: Update dynamic sensitivity. As 
explained above, the dynamic sensitivity represents 
deviations between the axes of the global orthogonal 
coordinate system and the axes of the pairs of accelerom- 
eters. Since the angular orientation of pen 10 may have 
changed, the dynamic sensitivity may also have changed. 

Given the change in the angular orientation of 
pen 10, the new dynamic sensitivity may be computed from 
the new angular orientation and the old matrix of dynamic 
sensitivity. 

STEP 460: Compute translational velocity and 
displacement. The translational velocity is computed by 
integrating the translational acceleration with respect 
to time. The displacement is computed by integrating the 
translational velocity with respect to time. 

STEP 470: Move screen object. Based on the 
output of previous steps which comprises translational 
acceleration, velocity and displacement as well as angu- 
lar acceleration, velocity and orientation, the screen 
object is moved. The moving of the screen object may be 
according to any appropriate transformation of the mo- 
tions of pen 10. 

Reference is now additionally made to Fig. 5A 
which is a simplified flowchart illustrating to operation 
of step 440 of Pig. 4. Fig. 5A includes the following 
steps: 

STEP 480: Set initial parameters. The rota- 
tional parameters comprise the three Euler angles; the 
three angular velocities; and the three angular accelera- 
tions. The initial value for the Euler angles is comput- 
ed based on the previously known value of the parameters, 
assuming that the acceleration has remained constant. 

STEP 482: Compute differential acceleration 
from model. First the position of an accelerometer in 
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the coordinate system of the pen is defined by vector r 
and the rotation of the pen in the global coordinate 
system is defined by a rotation matrix A(phi). For 
example, A (phi) may be an appropriate rotation matrix as 
presented in sections 14.10-5 through 14.10-7, pages 475- 
48*0 of Mathematical Handbook for Scientists and Engineers 
by Korn and Korn, 2nd Edition, published by McGraw-Hill 
in 1968. Here phi is a vector of Euler angles: 
phi=(a, £,r) T . Then the position of the accelerometer in 
the global coordinate system R is defined by R=Ar. 

Reference is hereby additionally made to Fig. 
5B, which contains equations useful for understanding the 
steps of Pig. 5A. Equation 490 illustrates the computa- 
tion of the acceleration of the accelerometer in the 
global coordinate system. 

As the Euler angles of the accelerometer 
change, the sensitivity vector K also changes. The 
change in the sensitivity vector K may be computed by 
using equation 492 of Fig. 5B. 

Given the acceleration of the accelerometer and 
the new sensitivity vector, the estimated value for the 
differential signal of the accelerometer, u est may be 
computed by using equation 494 of Fig. 5B.' 

The remainder of the parameters may be computed 
with an appropriate model. Preferably, a model which 
allows the use of only the parameters specified above, 
rather than a larger number of parameters, is used. For 
example,, equation 496 of Fig. 5B represents an appropri- 
ate model for computing the remainder of the parameters. 

STEP 484: Is the difference between the com- 
puted and the current value less than a predetermined 
amount? If the difference is less than this amount, the 
estimated parameters are taken to be correct and itera- 
tion is terminated, with the computed parameters being 
reported. 

An appropriate value for the predetermined 
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amount may vary depending on, for example, the maximum 
number of desired iterations. One possible appropriate 
value would be .0003 g, where g represents the accelera- 
tion of gravity at the earths surface. 

STEP 486: Compute changes in estimated angles 
according to the gradient method. New estimated angles 
are computed by adding a change to the old estimated 
angles; the change is computed according to the gradient 
method. The gradient method is explained more fully in 
section 20.3-3 of Mathematical Handbook for Scientists 
and Engineers by Korn and Korn, referred to above. 

STEP 488: Compute new parameters. New values 
for the remaining parameters are computed. Iteration 
then continues with step 482. 

The functionality of the apparatus of Fig. l 
when performing handwriting recognition will now be 
briefly described. Pen 10 sends acceleration data 
through . control circuit 40 to CPU 50. Teaching and 
recognition then occur based on the data from pen 10. 

Reference is now additionally made to Fig. 6 
which is a simplified block diagram of the apparatus of 
Fig. 1 when used for handwriting recognition. The appa- 
ratus of Fig. 6 receives input from pen 10. 

Pen 10, when moved by the user of the handwrit- 
ing recognition apparatus, transmits data describing the 
accelerations of pen 10 over time to acceleration teach- 
ing control 630 and/or acceleration handwriting recogni- 
tion control 650. 

The data from pen 10 may be transmitted to 
acceleration teaching control 630. Transmission to accel- 
eration teaching control 630 typically occurs for each 
person who is to use the system for handwriting recogni- 
tion for the first time. Transmission to acceleration 
teaching control 630 also preferebly occurs when recogni- 
tion errors are detected; use of acceleration teaching 
control 630 when recognition errors are detected is 



WO 95/21436 



PCT/US95/01483 



16 

termed herein adaptive teaching. 

Acceleration teaching control 630 operates on 
the data received, which data represents hand movements 
by the user when writing a symbol, together with manual- 
ly-provided identification of the symbol codes that are 
associated with the data. Acceleration teaching control 
630 then updates database 640, a per-person per-symbol 
acceleration database. Database 640 comprises prototypes 
of accelerations for each symbol, comprising data specif- 
ic to each person for each symbol. 

Alternatively, the data from pen 10 may be 
transmitted to acceleration handwriting recognition 
control 650. Acceleration handwriting recognition con- 
trol 650 operates on the data received from pen 10 to 
recognize the symbol represented by the movement of pen 
10. 

The output of acceleration handwriting recogni- 
tion control 650 comprises a list of symbol codes and 
their respective probabilities. An acceleration hand- 
writing recognition post-processing circuit 660, chooses 
the correct symbol code based on the list of symbol codes 
and probabilities, and on post-processing information 
which preferably comprises a database of previous confu- 
sions and a dictionary. The output of acceleration hand- 
writing recognition post-processing circuit 660 is a list 
of symbol codes and/or words sorted by likelihood. 

Reference is now additionally made to Figs. 7A 
and 7B which are simplified flow charts illustrating 
operation of the apparatus of Fig. 7 in accordance with a 
preferred embodiment of the invention, when performing 
handwriting recognition. Fig. 7A illustrates the teach- 
ing process and Fig. 7B illustrates the recognition 
process. The steps in Fig. 7A include the following: 

STEP 710: Read accelerometer data. The accel- 
erometer data comprises data points representing sampling 
of the acceleration measured by accelerometers 25. 
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Preferably, the sampling rate is approximately 1600 data 
points per second, averaged over 8 points, producing an 
output of approximately 200 data points per second. 

STEP 712: Identify pen-surface contact termi- 
nation. The data from step 710 does not include the 
surface contact status of pen 10. The surface contact 
status of pen 10 may be derived from the acceleration 
data as follows: 

The acceleration data is filtered to remove 
components other than noise. For example, the accelera- 
tion data may be filtered by a Butterworth digital filter 
described in Digital Filter Design by T.W. Parks and C.S 
Burrus, ' John Wiley & Sons, 1987, chapter 7, section 
7.3.3, using the 4th order lowpass digital filter with a 
cut-off frequency of 0.7 to 0.9. 

The filtered acceleration data is then inte- 
grated over time. The slope of the integrated filtered 
acceleration data is then analyzed to determine the point 
at which the slope exceeds a threshold value. The point 
at which the slope exceeds the threshold value is taken 
to be the first point with status "pen down". The point 
at which the slope falls below a threshold value is taken 
to be the first point with status "pen up"; the threshold 
value may or may not be the same as the previously de- 
scribed threshold value. 

The threshold values described above may be 
determined in advance for the particular type of pen and 
writing surface, may be determined by a learning process 
for the particular person, or may be determined by other 
means. 

STEP 715: Identify individual symbols and 
words. The data from the previous step is divided into 
data representing individual symbols. The status which 
comprises the status of "pen up" is termed herein "pen 
not down". Preferably, the number of consecutive data 
points with status of "pen not down", which data points 
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represent a particular duration of the status "pen not 
down" is taken to indicate the end of a symbol or of a 
word. 

Typically, the duration of status "pen not 
down" within a range from 200 milliseconds to 400 milli- 
seconds is taken to indicate the end of a symbol. Dura- 
tion of the status "pen not down" in the range from 800 
milliseconds to 1200 milliseconds is typically taken to 
indicate the end of a word. 

Alternatively, the end of a symbol or of a word 
may be indicated by data points which represent pen 
movements that are not part of a symbol, or by other 
means. Output data from step 715 comprises symbol end 
and word end data. 

STEP 720: Normalize accelerometer data. The 
acceleroraeter data is normalized in time or by other 
means. Appendix A is a computer listing in the C pro- 
graming language comprising routines that are a preferred 
implementation of step 720. The routines comprise the 
following routines in section II, "pre-preprocessing" : 
normal; together with various definitions used by routine 
normal . 

STEP 730: Filter accelerometer data. The 
normalized accelerometer data received from the previous 
step is filtered in order to remove noise. The filtering 
may be accomplished by iterative smoothing of adjacent 
points until the total change in the signal due to a 
smoothing operation is less than the desired accuracy of 
the data, or by other suitable means. 

STEP 740: Parameterize accelerometer data. 
The data is parameterized according to criteria which are 
chosen .to represent each symbol. If the accelerometers 
are not mutually orthogonal, the acceleration data may be 
converted into equivalent data in a mutually orthogonal 
coordinate system as follows: 

Let the non-orthogonal signals be denoted by 
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the vector u=(u 1/ u 2 ,u 3 ) T and the orthogonal signals be 
denoted by the vector u , = (u , 1 ,u' 2/ u , 3 ) T . Then u'-AqA^u 
where A is a vector of static sensitivity vectors 
A=(A 1/ A 2/ A 3 ) of the three accelerometers. The static 
sensitivity vector is computed from the outputs of the 
accelerometers during a defined orientation without 
movement. A Q is a diagonalized matrix of sensitivity of 
the orthogonal coordinate system comprising the norms of 

The parameters preferably comprise the follow- 
ing: 

number of points before normalization; 

normalized signal of pen status; 

normalized signal of Z acceleration; 

sine of the angle or' which angle is defined as 
the angle between the vector associated with the current 
data point (AccX i/ AccY i# AccZ i ) and the AccXAccY plane as 
shown in Fig. 8A; 

cosine of the angle or'; 

sine of the angle £> 1 which angle is defined as 
the angle between the vector that connects the point 
before the previous data point (AccX^, AccY^, AccZ^) 
and the current point (AccX^, AccY^, AccZ^) , and the vector 
that connects the current point with the point after the 
subsequent point ( AccX i+2 , AccY i+2 , AccZ i+2 ) in space 
(AccX,AccY, AccZ) as shown in Fig. 8B; 

and cosine of the angle 

STEP 750: Generalize parameters. The parame- 
ters of the symbol being learned represent a specific 
instance of the symbol. The symbol prototype stored by 
the system is to represent the general characteristics of 
the symbol as drawn by that person. Therefore, the 
parameters of the symbol being learned are generalized by 
some suitable means, such as by computation of the aver- 
age of the value of each parameter from previous in- 
stances of the symbol along with the value of each param- 



Patent provigU^^ 



WO 95/21436 



PCT/US95/01483 



20 

eter from the current instance of the symbol, 

STEP 760: Update per-person per-syrabol accel- 
eration prototype database. The newly computed parame- 
ters from the previous step are stored in the per-person 
per-symbol acceleration prototype database. 

The steps in Fig. 7B include steps which have 
already been described above with reference to Pig. 7A. 
The remainder of the steps in Fig. 7B include the follow- 
ing: 

STEP 800: For each prototype in the per-person 
per-symbol acceleration prototype database, build a 
measure * of comparison between the sample and the proto- 
type, combined over parameters in the prototype. In 
accordance with a preferred embodiment of the present 
invention, all parameters are combined together to pro- 
duce the measure of comparison. Appendix A is a computer 
listing in the C programing language comprising routines 
that are a preferred implementation of step 800. The 
routines comprise the following, which are found in 
section V, "symbols recognition": make_corr; correljiem; 
obj_funct; together with various definitions used by the 
routines . 

STEP 810: Create a list of probable symbols 
sorted by likelihood. Based on the measure or measures 
of comparison generated in step 800, a single list of 
probable symbols sorted by likelihood is generated. 

STEP 820: Choose the correct symbols and the 
correct word based on the list, the database of previous 
confusions and a dictionary. The symbols with greatest 
likelihood are the candidates from which the correct 
symbol is chosen. 

The database of previous confusions provides 
information that allows the correction of the choice of 
the correct symbol based on previous incorrect identifi- 
cations. The database of previous confusions comprises, 
for each symbol, a list of other symbols which have been 
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confused with the first symbol; for example, that the 
symbol "f" has often been confused with the symbol "b". 
When such an entry is found comprising previous confu- 
sions for a symbol in the list, the symbol or symbols 
that have previously been confused with the symbol in the 
lifct are added to the list. In accordance with the 
previous example, if the symbol "f" is found in the list, 
then the symbol "b" is added to the list. 

An indication of the end of each word has been 
passed as output since step 715, described above. Based 
on the indication, the most likely word, comprising the 
most likely identifications for each symbol in the list, 
is identified. 

The most likely word is checked against the 
dictionary. Preferably, the dictionary comprises both a 
general dictionary used for all users of the system and a 
personal dictionary for each user of the system. If an 
entry exists in the dictionary for the most likely word, 
the word is chosen as the correct identification. 

If the most likely word is not found in' the 
dictionary, all possible word combinations in the list 
are formed and each is checked against the dictionary. 
Among all such words which are found in the dictionary, 
the word with the highest likelihood is then chosen as 
the correct identification. 

If none of the words is found in the diction- 
ary, the most likely word is chosen as the correct iden- 
tification. 

STEP 830: Check to see if a correction has 
been entered. During the process of recognition, the 
user of the system is preferably provided with a visual 
indication of each symbol recognized. 

After the end of a word is detected, the user 
of the system preferably is provided with a visual indi- 
cation of the word recognized. The user may indicate 
manually that a given word was incorrectly recognized and 
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may input a correction. 

STEP 840: Update database of previous confu- 
sions. Based on a manual correction entered in step 830 
or an automatic correction based on the dictionary, the 
database of previous confusions is updated. Based on a 
manual correction, the personal dictionary is also updat- 
ed if the corrected word is not found in the dictionary. 

Preferred methods and apparatus for handwriting 
recognition are described in the following applications, 
the disclosure of which is hereby incorporated by refer- 
ence: PCT/US92/08703; Israel 104575; PCT application 
filed 31 January 1994 in the US Receiving Office by Ehud 
Baron and Edward A. Wolfe. 

It is appreciated that the particular embodi- 
ment described in Appendix A is intended only to provide 
an extremely detailed disclosure of the present invention 
and is not intended to be limiting. 

It is appreciated that various features of the 
invention which are, for clarity, described in the con- 
texts of separate embodiments may also be provided in 
combination in a single embodiment. Conversely, various 
features of the invention which are, for brevity, de- 
scribed in the context of a single embodiment may also be 
provided separately or in any suitable subcombination. 

It will be appreciated by persons skilled in 
the art that the present invention is not limited to what 
has been particularly shown and described hereinabove. 
Rather, the scope of the present invention is defined 
only by the claims that follow: 
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APPENDIX A 
Recognition according a combination of signals 



Definitions and data structure 

Board. H tils 

r Function iniLdatatr ( portbase ) sets communication with data 
translation board via port portbase. It returns : 

0 - communication was established ; 
-1 - error on board (board is not exist). */ 
//int init_datatr ( int ) ; 

int newcomp ( void ) ; 

int read_ch ( int channel , int gain ) ; 

//int read_point ( struct point * , int ) ; 

/*int read_block ( struct point * , int max_numb_point , 

int timeoutJor_begin , int timeout_for_end , 
int key_mouse_stop ) ; */ 

//int read_symbol ( struct point • , int f int ) ; 

int mshit ( void ) ; 

void ciose_datatr ( void ) ; 

#define PORT_BASE 0x210 

#define KEY_STOP 0x1 
#define MOUSE_STOP 0x2 
#define KEY_MOUSE_STOP 0x3 

#define PENJ/VAIT 0x1 
#define PEN.NOWAIT 0x0 



EtafaH file 
struct point pen 
{ 

unsigned ax ; 
unsigned ay ; 
unsigned az ; 
unsigned pn ; 

}; 

struct pointjablet 

{ 

int x ; 
inty ; 
int p ; 

};. 

#define SYNCROBIT 0x80 
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Datar.H file 




^define PORT_BASE 0x210 


#define CSR 


0x0 


^define GAIN 


0x1 


^define DAC0_LOW 0x2 


#define DACO.HIGH 0x3 


#define DAC1_L0W 0x4 


#define DAC1_HIGH 0x5 


#define CHANNEL_AX 0x4 


#define CHANNEL_AY 0x5 


#define CHANNEL_AZ 0x6 


#define CHANNEL 


..PN 0x7 


#define STATUS 


Oxe 


#define CHANNEL_EMPTY 0x0 


#define ID REGISTER Oxf 


#define GAINjt 


0x00 


#define GAlN_2 


0x40 


#define GAIN_4 


0x80 


#define GAIN_8 


OxcO 


#define IER 


0x21 


#define IIR 


0x20 


#define IRGO 


0x08 


#define IRQ1 


0x09 


^define IRQ2 


OxOa 


#define IRQ3 


OxOb 


^define IRQ4 


OxOc 


#define IRQ5 


OxOd 


#define IRQ6 


OxOe 


#define IRQ7 


OxOf 


/•struct point { 




unsigned ax ; 




unsigned ay ; 




unsigned az ; 




unsigned pn ; 




};*/ 




#define MINUS.PEN 1700 


r#define PEN.UP 


0x2 



#define PEN_DOWN 0x4 
#define PENJTHRSLD 200 
#define EMPTY o */ 

#define BUFSI2E 0x80 



^define TIME_COUNT 3000 
#inciude <dos.h> 
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Ser,H file 

r * 

FILENAME: SERJALH 

Some definitions used by SER.C 
. v 

#define COM1 1 
#defme COM2 2 

#defme COM1BASE 0x3F8 r Base port address for COM1 7 
#define CQM2BASE 0x2F8 /* Base port address for COM2 V 

r 

The 8250 UART has 10 registers accessible through 7 port addresses. 
Here are their addresses relative to COM1BASE and COM2BASE. Note 
that the baud rate registers, (DLL) and (DLH) are active only when 
the Divisor-Latch Access-Bit (DLAB) is on. The (DLAB) is bit 7 of 
the (LCR). 

o TXR Output data to the serial port, 
o RXR Input data from the serial port, 
o LCR Initialize the serial port, 
o iER Controls internet generation, 
o IIR Identifies Internets, 
o MCR Send contort signals to the modem, 
o LSR Monitor the status of the serial port, 
o MSR Receive status of the modem, 
o DLL Low byte of baud rate divisor, 
o DHH High byte of baud rate divisor. 



7 

#define TXR 


0 


r Transmit register (WRITE) 7 


#define RXR 


0 


r Receive register (READ) 7 


#define IER 


1 


r Interrupt Enable 


V 


#define IIR 


2 


r Interrupt ID 


V 


#define LCR 


3 


r Line control 


V 


define MCR 


4 


r Modem control 


V 


* define LSR 


5 


/♦ Line Status 


V 


^define MSR 


6 


r Modem Status 


V 


#define DLL 


0 


1* Divisor Latch Low 


7 


#define DLH 


1 


f Divisor latch High 


7 



/• 

Bit values held in the Line Control Register (LCR). 



bit 


meaning 


0-1 


00=5 bits, 01=6 bits, 10=7 bits, 11=8 bits. 


2 


Stop bits. 


3 


0=parity off, 1=parity on. 


4 


0=parity odd, 1=parity even. 


5 


Sticky parity. 


6 


Set break. 


7 


Toggle port addresses. 



#define MOLARITY 0x00 
#define EVEN.PARITY 0x18 
#define OL .. .PARITY 0x08 
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Bit values held in the Line Status Register (LSR). 
bit meaning 



0 
1 
2 
3 
4 

5 s 
6 

7 



Data ready. 

Overrun error - Data register overwritten. 
Parity error - bad transmission. 
Framing error - No stop bit was found. 
Break detect - End to transmission requested. 
Transmitter holding register is empty. 
Transmitter shift register is empty. 
Time out - off line. 
— •/ 



#define RCVRDY 


0x01 


#define OVRERR 


0x02 


#define PRTYERR 


0x04 


#define FRMERR 


0x08 


#define BRKERR 


0x10 


#define XMTRDY 


0x20 


#define XMTRSR 


0x40 


#define TIMEOUT 





0x80 



Bit values held in the Modem Output Control Register (MCR). 
bit meaning 



0 
1 
2 
3 



4 

5-7 



#define DTR 0x01 
#define RTS 0x02 
#define MCJNT 



Data Terminal Ready. Computer ready to go. 
Request To Send. Computer wants to send data, 
auxiliary output #1 . 

auxiliary output #2.(Note: This bit must be 

set to allow the communications card to send 

interrupts to the system) 

UART ouput looped back as input. 

not used. 

7 



0x08 



Bit values held in the Modem Input Status Register (MSR). 

bit meaning 

0 delta Clear To Send. 

1 delta Data Set Ready. 

2 delta Ring Indicator. 

3 delta Data Carrier Detect. 

4 Clear To Send. 

5 Data Set Ready. 

6 Ring Indicator. 

7 Data Carrier Detect. 



#define CTS 0x10 
#define DSR 0x20 
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/• . 

Bit values held in the Interrupt Enable Register (IER). 
bit meaning 

0 Interrupt when data received. 

1 Interrupt when transmitter holding reg. empty. 

2 Interrupt when data reception error. 

3 Interrupt when change in modem status register 
4-7 Not used. 



#define RXJNT 0x01 



•7 



Bit values held in the Interrupt Identification Register (MR), 
bit meaning 

0 Interrupt pending 

1-2 Interrupt ID code 

00=Change in modem status register, 
01=Transmitter holding register empty, 
10=Data received, 

11 deception error, or break encountered. 
3-7 Not used. 

* 7 

#define RXJD 0x04 
#define RXJvlASK 0x07 



r 

These are the port addresses of the 8259 Programmable Interrupt 
Controller (PIC). K 

7 

#define IMR 0x21 r Interrupt Mask Register port 7 

#definelCR 0x20 /* Interrupt Control Port 7 



r 

An end of interrupt needs to be sent to the Control Port of 
the 8259 when a hardware interrupt ends. 

#defineEOI 0x20 r End Of Interrupt 7 

r 

The (IMR) tells the (PIC) to service an interrupt only if it 
is not masked (FALSE). 

#define IRQ3 0xF7 T COM2 7 

#define IRQ4 OxEF r COM1 7 



r 

The (IMR) tells the (PIC) to service an interrupt only If it 
is not masked (FALSE). 

7 

^define IRQ3 0xF7 r COM2 7 

#define IRQ4 OxEF r COM1 7 
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int flag; LC 
int SetSerial(); 

int SetOthers(int Parity, int Bits, int StopBit); 
int SetSpeed(int Speed); 
int SetPort(int Port); 
void init_serial(void); 
void comm_off(void); 

void setaIIport(int Port, int Speed, int Parity, int Bits, int StopBit); 
int putchport (char); 
void putstrport(char *); 
int getchport(void); 
void offport(); 



Serconst.H file 
FILENAME: SERCONST.H 

Some definitions used by SER.C 



r 

The 8250 UART has 10 registers accessible through 7 port addresses. 
Here are their addresses relative to COM 1 BASE and COM2BASE. Note 
that the baud rate registers, (DLL) and (DLH) are active only when 
the Divisor-Latch Access-Bit (DLAB) is on. The (DLAB) is bit 7 of 
the (LCR). 

o TXR Output data to the serial port, 
o RXR input data from the serial port, 
o LCR initialize the serial port. . 
o IER Controls interrupt generation, 
o IIR Identifies interrupts, 
o MCR Send contorl signals to the modem, 
o LSR Monitor the status of the serial port, 
o MSR Receive status of the modem, 
o DLL Low byte of baud rate divisor, 
o DHH High byte of baud rate divisor. 

V 

#define TXR 0 r Transmit register (WRITE) 7 

#define RXR 0 t Receive register (READ) •/ 

#define IER 1 r interrupt Enable V 

#define IIR 2 r Interrupt ID V 

#define LCR 3 r Line control V 

#define MCR 4 r Modem control V 

#define LSR 5 r Line Status V 

#define MSR 6 r Modem Status V 

#define DLL 0 r Divisor Latch Low V 

#define DLH 1 /* Divisor latch High 7 

#define DLAB 0x80 p V 

r . 



Bit values held in the Line Control Register (LCR). 
bit meaning 
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0-1 00=5 bits, 01=6 bits, 10=7 bits, 11=8 bits. 

2 Stop bits. 

3 0=parity off, 1=parity on. 

4 0=parity odd, 1=parity even. 

5 Sticky parity. 

6 Set break. 

7 Toggle port addresses. 

• •/ 

#define NO_PARITY 0x00 
#define EVEN.PARITY 0x1 8 
#define ODD_PARITY 0x08 



r * 

Bit values held in the Line Status Register (LSR). 
bit meaning 

0 Data ready. 

1 Overrun error - Data register overwritten. 

2 Parity error - bad transmission. 

3 Framing error - No stop bit was found. 

4 Break detect - End to transmission requested. 

5 Transmitter holding register is empty. 

6 Transmitter shift register is empty. 

7 Time out - off line. 

*/ 

#define RCVRDY 0x01 

#define OVRERR 0x02 

Sdefine PRTYERR 0x04 

#define FRMERR 0x08 

#define BRKERR 0x10 

#define XMTRDY 0x20 

#define XMTRSR 0x40 
#define TIMEOUT 0x80 

r * 

Bit values held in the Modem Output Control Register (MCR). 
bit meaning 

0 Data Terminal Ready. Computer ready to go. 

1 Request To Send. Computer wants to send data. 

2 auxiliary output #1 . 

3 auxiliary output #2.(Note: This bit must be 
set to allow the communications card to send 
interrupts to the system) 

4 UART ouput looped back as input. 
5-7 not used. . 

* •/ 

#define DTR 0x01 
#define RTS 0x02 
#define MCJNT 0x08 



Bit values held in the Modem Input Status Register (MSR). 
bit meaning 
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0 


delta Clear To Send. 


1 


delta Data Set Ready. 


2 


delta Ring Indicator. 


3 


delta Data Carrier Detect. 


4 


Clear To Send. 


5 


Data Set Ready. 


6 


Ring Indicator. 


7 


Data Carrier Detect. 



#define CTS 0x10 
#define DSR 0x20 



Bit values held in the Interrupt Enable Register (IER). 
bit meaning 

0 Interrupt when data received. 

1 Interrupt when transmitter holding reg. empty. 

2 ' Interrupt when data reception error. 

3 Interrupt when change in modem status register. 
4-7 Not used. 

* v 

#define RXJNT 0x01 



Bit values held in the interrupt Identification Register (IIR). 
bit meaning 

0 Interrupt pending 

1-2 Interrupt ID code 

00=Change in modem status register, 
01=Transmitter holding register empty, 
10=Data received, 

11=reception error, or break encountered. 
3-7 Not used. 
• _ 

#define RXJD 0x04 
#define RX_MASK 0x07 



r 

These are the port addresses of the 8259 Programmable Interrupt 
Controller (PIC). 

7 

#define IMR 0x21 r Interrupt Mask Register port 7 

#define ICR 0x20 t Interrupt Control Port 7 



r 

An end of interrupt needs to be sent to the Control Port of 
the 8259 when a hardware interrupt ends. 

7 

#defineEOI 0x20 r End Of Interrupt 7 
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The (IMR) tells the (PIC) to service an interrupt only if it 
is not masked (FALSE). 

V 

/•unsigned char IRQ[8] = { -0x01 , -0x02 , -0x04 , -0x80 , 

-0x10 , -Ox V 
#define IRQ3 0xF7 r COM2 •/ 

#define IRQ4 OxEF r COM1 7 

int SerSetPortBase ( int , unsigned • ) ; 

int SerSetSpeed ( unsigned , long ) ; 

int SerSetBitsParityStopBit ( unsigned , int , int , int ) ; 

int SerPutChar ( unsigned , unsigned char ) ; 

int SierPutSlring ( unsigned , unsigned char * ) ; 

int SerinitBuffer ( unsigned ) ; 

int SerGetChar ( unsigned ) ; 

int SerTestDSR ( unsigned ) ; 

int SerTestCTS ( unsigned ) ; 

r int flag; 
int SetSeriaiQ; 

int SetOthers(int Parity, int Bits, int StopBit); 
int SetSpeed(int Speed); 
int SetPort(int Port); 
void iniLserial(void); 
void comm_off(void); 

void setailport(int Port, int Speed, int Parity, int Bits, int StopBit); 
int putchport (char); 
void putstrport(char •); 
int getchport(void); 
void offportQ; 
V 



Tablet.H file 

#define PENJDOWN 1 

#define PENJJP o 

#define PEN.OUTPROX 99 

#define TBLJ/VACOMJI 3 

#define TBL_DATA_ASCII 1 

#define TBL_DATA_BINARY 0 

#define TBL_MODE_STREAM 3 
#define TBL_MODE_SWITCI-LSTREAM 2 

#define TBL_MODE_SUPRESSED O 

#define TBL_MODE_PO!NT 0 

#define TBL.TYPE.ABSOLUTE 0 

#define TBLJTYPE^RELATIVE 1 

#define TBL_MlLLIMETERS 0 

#define TBLJNCHES 1 

#define TBL_ALWAYS_TRANSMIT_YES 1 

#define TBLJ^LWAYS.TRANSMITJSIO 0 

#define TBL_BAUD_1 9200 7 

#define TBL_BAUD_9600 6 

#define TBL_BAUD_4800 5 

#define TBL_BAUD_2400 4 

#define TBLJAUD Jl 200 3 

#define TBL_BAUD_6O0 2 

#define TBL_BAUD_300 1 
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#define TBL_BAUD_150 o 
#define TBL_PARITY_NONE o 
#define TBL_PARITY_ODD 1 
#define TBL_PARITY_EVEN 2 
#define TBL_STOPBITS_1 o 
#define TBL_STOPBITS_2 1 
#define TB L_DS R_M ON ITO R_OFF o 
#define TBL_DSR_MONITOR_ON 1 
#define TBL_DATALENGTH_7 0 
#define TBL_DATALENGTH_8 1 
#define TBL_TRANSFER_RATE_MAX 7 
#define TBL_TRANSFER_RATE 100 6 
#define TBL_TRANSFER_RATeI67 5 
#define TB L_TRANS FER_RATE_50 4 
^define TBL_TRANSFER_RATE 20 3 
#define TBL_TRANSFER_RATEll 0 2 
#define TBL_TRANSFER_RATE 5 1 
#define TBL_TRANSFER_RATEll o 
#define TB L_OR I G I N LOG_U P PER_LEFT 1 
#define TBLJ)R!GINLOG_LOWER LEFT 0 
#define TBL.DATA.TERMINATOrIcR^LF 2 
#define TBL_DATA_TERMINATOR_LF 1 
#define TBL_DATAJTERM!NATOR_CR 0 

int read _point_tableLpen ( unsigned , int , 

struct poinUablet * , struct poinLpen *[8] ) ; 
int find_seLparametersJablet ( int comport , unsigned 'portbase ) ; 
int init_tablet ( int port , unsigned 'portbase , int command_set , 



int datajormat , int operation_mode , int originjype , 
int unit_mesure , int a!ways_transmit , int speed , 
int parity , int stopbit , int dsr_monitor , 
int datalength , int transfer_rate , int crigjog , 
int data_terminator , int max_x , int max.Ty ) ;' 



void ciose_tablet ( unsigned portbase ) ; 
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I. Reading from device 
r This procedure reads synchronized data from the graphic tablet and accelerometers 

int read_point_tablet_pen ( unsigned portbase , int read_pen , 

struct poinUabiet tablet . 
^ struct point_pen pen[8] ) 

int ind_package = 0 , reply , debug[10] , i ; 

unsigned char package[7] = {0,0,0,0,0,0,0}; 

if ( read_pen ) 

read_point_pen ( &pen[0] ) ; 
i=0; 

r Waiting for synchro-bit */ 

do 

{ 

if ( ( reply = SerGetChar { portbase ) ) < 0 ) 

return reply ; 
debug[i++] = reply ; 

if ( ( package[0] = (char) reply ) & SYNCROBIT ) 
break ; 

} while ( ind_package++ < 10 ) ; 

r Error - No synchro-bit in 10 bytes 7 

if ( ind_package >= 10 ) 
return SER_SYNCROBIT ; 

r Read the next 6 bytes from tablet and 6 points from accelerometer 7 
for ( ind_package = 1 ; ind_package < 7 ; ind_package++ ) 

if ( read_pen ) 

{ 

^ read_point_pen ( &pen[ind_package] ) ; 

if ( ( reply = SerGetChar ( portbase ) ) < 0 ) 
return reply ; 
^ package[ind_package] = (char) reply ; 

r Read last point from accelerometer 7 

if ( read_pen ) 
read_point_pen ( &pen[ind_package] ) ; 

/* Calculates the values of the signals for tablet 7 

tablet->x = ( package[0] & 0x03 ) « 14 ; 
tablet->x ( package[1] & 0x7f ) « 7 ; 
tablet->x += ( package[2] & 0x7f ) ; 
if ( packaged] & 0x04 ) 

tablet->x = - tablet->x ; 
tablet->y = ( package[3] & 0x03 ) « 14 ; 
tablet->y +« ( package[4] & 0x7f ) « 7 ; 
tablet->y += ( package[5] & 0x7f ) ; 
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tablet->p = 0 ; 

if ( ! ( package[0] & 0x40 ) ) 

tablet->p = 99 ; 
if ( package[3] & 0x04 ) 

tablet->y = - tablet->y ; 
if ( package[6] & 0x20 ) 

tablet->p = ( package[6] & 0x1 f ) ; 
return 0 ; 

} 



11. Pre-processing 

/• Two procedures: Normalization in time and filtering the input signals by smoothing 

void normal ( int num_o!d , float arr_old[] , int num_new , float arr_newfl ) 

double koeff ; 

int ind_old , indjiew ; 

koeff s (double) { num_old - 1 ) / (float) ( num_new - 1 ) ; 
arr_new[0] = arr_old[0] ; 

for ( ind^new = 1 ; ind.new < num_new - 1 ; ind_new ++ ) { 
ind_old = (int) ( floor ( koeff * ind_new ));'"" 
arr_new[ind_new] = ( ind_oid + 1 - koeff * ind_new ) ■ arr_o!d[ind_old] + 

( koeff • ind^new - ind_old ) * arr_o!d[ind_old + 1] ; 
arr_newpnd_new] « arr new[ind_new] ; 

} 

arr_new[ind_new] = arr_old[num_old-1] ; 



float smoothl ( int num , float zQ ) 

int ind ; 
float temp ; 
float norma ; 

for (ind = 1 , norma = 0 ; ind < num - 1 ; ind++ ) { 
temp = ( z[ind -1]+z[ind]+z[ind+1] ) /3. ; 
norma += abbs ( zfind] - temp ) ; ■ 
z[ind] = temp ; 

} 

return norma ; 
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ill. Parameter's extraction 

/* Calculation of the parameters of a symbol from the input signals 7 

int make_par ( char arg_ch ) 
{ 

struct point { 

unsigned int x : 12 ; 

unsigned inty : 12 ; 

unsigned int 2 : 12 ; 

unsigned int pen : 4 ; 
} point , points[500]; 

int read_-oext_symboi ( FILE * , struct pointQ ) ; 

char file_name[40] ; 

int len t number_points ■ 0 ; 

FILE *injiie , •outjiie[10] , 'outjetter , *out_bin ; 

float param[6][NUMBER_POINT) , sum_par[6][NUMBER_POINT| ; 

int index = 0 , max_point ; 

int ind , start ; 

int cur_x , cur_y , cur_z , cur_p ; 

float arr_x[M AX_P OI NT] , an~y[MAX_POINT] , arr_z[MAX_POINT| , arrj3[MAX_PO!NT] ; 
r initialization of the results arrays to zero 7 

for ( ind = 0 ; ind < 6 ; ind++ ) 
for ( index = 0 ; index < NUMBER_?OINT ; index++ ) { 
parampnd][index] = 0.0 ; 
sum_par[indj[index] = 0.0 ; 

} 

r Identification of the file of data 7 

sprintf ( filename , ■%03d.smb - , (int) arg_ch ) ; 
if ( ( injile = fopen ( fiie.name , °rb" ) ) — NULL ) 

{ 

strcpy (exLerr,file_name); 
return -4 ; 

} 

start = 0 ; 

f Reading data from file 7 

while ( ( max_point = read_next_symboI ( injile , points ) ) > 0 ) { 
for ( index = 0 ; index < max_point ; index++ ) { 
arr_x[index] = (float) points[index].x ; 
arr_y[index] = (float) pointspndexj.y ; 
arr_z[index] = (float) points[index].z ; 
arr_p[index] = (float) points[index].pen ; 

arr_p[0] = arr_p[max_point - 1] = 1 ; 

start++ ; 

number_points += max_point ; 
r Calling the procedure make_par_let for calculating parameters 1-6 7 
make _parjet ( arr_x , arr.y , arr_z , arr_p , param , max_point - 1 ) ; 




WO 95/21436 



36 



PCT/US95/01483 



/• Calculating the average of each parameter */ 

for ( ind = 0 ; ind < 6 ; ind-H- ) 
for ( index = 0 ; index < NUMBER_POINT ; index++ ) { 
sum_par[ind][index] += param[ind][index] ; 

) 

) 

for ( ind = 0 ; ind < 6 ; ind++ ) 
for ( index = 0 ; index < NUMBER.POINT ; index++ ) 
sum_par[ind][index] /= start ; . 

sum_par[0][0] = (float) number_points / start ; 
fclose ( in«Jle ) ; 

r write avg in Binary file V 
sprintf ( filename , ,, %03d.par M , (int) arg_ch ) ; 
outjetter = fopen ( filename , B wb+ B ) ; 
for ( index = 0 ; index < 6 ; !ndex++ ) 
fwrite ( sum_par[index] , sizeof(float) , NUMBER_POlNT , outjetter); 

fciose ( outjetter ) ; 
return start ; 



void make_parjet ( float arr_xfl , float arr_yQ , float arr_zfl . 

^ float arrjjQ , float param[6][NUMBERlpOIN7] , int max_point ) 

float end_smooth; 

float new_arr_x[500] , new_arr_y[500] , new_arr_z[500] , new arr p[500] • 
int ind , index ; 

r Call for pre-processing 7 

norma] ( max_point , arr_x , NUMBER_POlNT , new_arr_x ) ; 
normal ( max_point , arr_y , NUMBER_PO!NT , newlarrly ) ; 
normal ( max_point , arr_z , NUMBER_POINT , new_arrlz ) ; 
normal ( max_point , arrjD , NUMBER_POINT , new~arrlp ) ; 
max_point = NUMBER_POlNT ; 
for ( ind = 0 ; ind < max_point ; ind++ ) { 

arr_x[ind] = new_arr_x[ind] ; 

arr_y[ind] = new_arr_y[ind] ; 

arr_z[ind] = new_arr_z[ind] ; 

arr_p[ind] = new_arr p[ind] ; 

} 

while ( ( end_smooth = smoothl ( max_point , arr_x ) ) > NUMBER_POINT / 10 ) 
while ( ( end_smooth = smoothl ( max_point , arr_y ) ) > NUMBER_PO!NT/ 10 ) 
while ( ( end.smooth = smoothl ( max__point , arr_z ) ) > NUMBER_PO!NT/ 10 ) 

/* Initialization of parameters 7 
param[0][0] = (float) arr_p[0] ; 
param[1][0] = ( arr_z[0] - arr_z[0] ) ; 
param[2][0] = 0.0 ; 
param[3][0] = 0.0 ; 
param[4][0] = 0.0 ; 
param[5][0] = 0.0 ; 
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param[0][1] = (float) arr_p[1] ; 

/* Calculation of parameters 7 
param[1][1] = ( arr_z[1] - arr_z[0] ) ; 

elev ( arr_x[2] - arr_x[0] , arr_y[2] - arr_y[0] , arr_z[2] - arr_z[0] , 

&param[2][1] , &param[3][1] ) ; 
param[4][1] = 0.0 ; 
param[5][1] = 0.0 ; 

for ( index = 2 ; index < max_point - 2 ; index++ ) { 
param[0][index] = (float) arr_p[index] ; 
param[1][index] = ( arr_z[index] - arr_z[0] ) ; 

elev ( arr_x[index + 1] - arr_x[index - 1] , arr_y[index + i] - arr_y[index - 1] f arr zflndex + 1] 
arr_z[indev 1] . 

&param[2][index] , &param[3][index] ) ; 
angles ( arr_x[index + 2] - arr_x[index] , 

arr_y[index + 2] - arr_y[index] , 

arr_z[index + 2] - arr_z[index] , 

arr_x[index] - arr_x[index - 2] , 

arr_y[index] - arr_y[index - 2] , 

arr_z[index] • arr_z[index - 2] , 

&param[4][index] , &param[5][index] ) ; 
index = index ; 

} 

param[0][index] = (float) arr_p[index] ; 
param[1][index] = ( arr_z[index] - arr_z[0] ) ; 
ar? e z^d?x C 1] ndeX + 13 " ar - XtlndeX " 1] ' arr ^ndex + 1] - arr.yfindex - 1] , arr_z[index + 1] - 

&param[2][index] , &param[3][index] ) ; 
param[4][index] = 0.0 ; 
param[5][index] = 0.0 ; 
index-n- ; 

r Calculation of parameters for last point 7 
param[0][index] = (float) arr_p[index] ; 
param[1][index] = ( arr_z[index] - arr_z[0] ) ; 
param[2][indexj = 0.0 ; 
param[3][index] = 0.0 ; 
param[4][index] = 0.0 ; 
param[5][index] = 0.0 ; 

} 



r Procedure elev calculates the SIN and COS of the angle of elevation 7 
void elev ( float x , float y , float z , float *cos_ug , float *sin_ug ) 
float norma ; 

norma = (float) sqrt (x*x+y*y+z*z) ; 
if ( norma < .00001 ) { 

*cos_ug = 0.0 ; 

*sin_ug =* 0.0 ; 

return ; 

} 

*cos_ug = ( (float) sqrt ( x • x + y • y ) ) / norma ; 
*sin_ug = z / norma ; 
return ; 

} 
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/* Procedure angles calculates the SIN and COS of the angle P */ 

void angles ( float x1 , float yl , float z1 , float x2 , float y2 , float z2 , 
float - cos_ug , float *sin_ug ) 

{ 

float normal , norma2 , x3 , y3 , 23 ; 
normal = ( float ) sqrt ( x1 * x1 + y1 * yl + z1 * 21 ) ; 
norma2 = ( float ) sqrt ( x2 * x2 + y2 * y2 + 22 * z2 ) ; 
if { normal < .0001 II norma2 < .0001 ) { 

'cos_ug = 0.0 ; 

*sin_ug = 0.0 ; 

return ; 

) 

•cos_ug = ( x1 * x2 + y1 * y2 + 21 * z2 ) / normal / norma2 ; 
x3 = ( y1 * 22 - 21 * y2 ) ; 
y3 = ( x2 * 21 - x1 * 22 ) ; 
23 = ( x1 * y2 - x2 * y1 ) ; 

*sin_ug = ( (float) sqrt ( x3 * x3 + y3 * y3 + 23 * 23 ) ) / normal / norma2 « 
return ; 
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IV. Training procedures 

/• Procedure for preliminary teaching 7 

int first teach ( void ) 
{ 

FILE -fp ; 
FILE*fpout; 
int i; 

char buf[4] , NdxStr[4] , symbols[256] ; 
int ndx = 0 , max_symb = 0 ; 
int num_sym; 

comment, ("converting data files, please wair.0,1); 

if ( ( fp=fopen ( "symbols.dar , "r" ) ) == NULL ) 

strcpy (exLerr I 0 symbols.dat B ); 

hide_comment ("converting data files, please wait°,0); 

return (-4); 

} 

while ( fscanf ( fp , u %s B , but ) > 0 ) 

symbols[max_symb++] = buf[0] ; 
fclose ( fp ) ; 

fpout=fopen pexLadpVw - ); 

for ( ndx = 0 ; ndx < max_symb ; ndx++ ) { 
sprintf ( NdxStr , -%03d" , ndx ) ; 
if ( ( num_sym=make_par ( symbolsfndx] ) ) <= 0 ) 

hide_comment ("converting data files, please wait B ,0); 
return (num_sym); 

} 

else for (i-0;i<num_sym;i++) 
^ fprintf (f0out,"%c" > symbols[ndx]); 

fclose (fpout); 

hide_comment ("converting data files, please wait°,0); 
return (0); 

} 



r procedure for adaptation of prototypes 7 

float huge *aILpan[100] ; 
int firsLadap ( void ) 
{ 

float old_rec t new_rec ; 
int count=0 , temp ; 
char *text ; 
char str[80]; 

if ( ( temp = readjext ("try.txr, &text ) ) < 0 ) 

return ( temp ) ; 
read_param ( ) ; 

new_rec = recogn ( Try.pri" , text , 0 , 0 ) ; 
sprintf (str, B %3f-before adaptation 0 ,new_rec); 
comment (str,-1,1); 
do{ 
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if (new_rec < 0 ) { 
hide_comment (str,-1); 
while ( alLpar[temp] != NULL ) { 



farfree ( alLpar[temp++] ) ; 



return ((int) new_rec); 



if ( new_rec > .995 ) 

break ; 
old_rec = new_rec ; 

new_rec = recogn ( Iry.prr, text , 1 , 0 ) ; 
if (new.rec <0 ) { 

hide^comment (str,-1); 

while ( alLparpemp] != NULL ) { 
farfree ( aiLpar[temp++] ) ; 

return ((int) new_rec); 

} 

hide_comment (str,-1); 

sprintf (str,"%3f- in adaptation" ,new_rec); 

comment (str,-1,i); 

new_rec = recogn ( try.prP, text , 0 , 0 ) ; 

hide_comment (str,-1); 

sprintf (str ( "%3f-after adaption u ,new_rec); 

comment (str,-1,1); 

if (new_rec < 0 ) { 

hide.comment (str,-1); 
while ( a!Loar[temp] != NULL ) { 
farfree ( all_par[temp++] ) ; 

return ((int) new_rec); 



} while ( fabs ( old.rec - new_rec ) > .005 & count++ < 9 ) ; 
hide_comment (str,-1); 
farfree ( text ) ; 

while ( alLparftemp] != NULL ) { 
farfree ( alLpar[temp++] ) ; 

} 

return 0 ; 

} 
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struct point { 
unsigned int x : 12 ; 
unsigned int y : 12 ; 
unsigned int z : 12 ; 
unsigned int pen : 4 ; 

}; 

struct reply 
{ 

int ndx; 
float weight; 

}; 

float recogn ( char *file_pen , char 'text , int adapt , int words ) 

float old_rec , new_rec , probs[10][20] ; 
int count=0 ; 

char symbols[256] , buf[4] ; 
unsigned long ttt ; 
int max_symb; 

FILE *injile , *file_symb , ~temp_word ; 
int symb; 

unsigned long start^word , end word ; 
float param[6][NUMBER_PO!NT] ; 
int index = 0 , max_point ; 
struct reply *repl ; 
int temp; 
int Ngood=0; 

int ind , NumSymbols ,ndx; 
struct point symb_pnts [MAX_POlNT|; 



arr x[MAX_POlN7] , arr_y[MAX_POINTl . arr_z[M APPOINT] , arr.p[MAX.POIN"q ; 



char letters[1 0][20],dict_wrds[1 0][20J; 
int end_of_word=0; 
int wrdlen; 

float sum[10],maxsum t ndx_maxsum; 
char org_wrd[20],f_word[20]; 
int txt_width; 
int i; 

if ( ( fiie_symb = fopen ( "symbols.dat" , V ; 
strcpy (exLerr 1 * , symbols.dat M ); 
return (-4); 

} 

for (ind=0;ind<256;ind++) map[ina>-1; 
max_symb ■ 0 ; 

while ( fscanf ( fi!e_symb , "%s B , buf ) > 0 ) 

map [bufIO]]=max_symb; 
symbols[max_$ymb++] = buf[0] ; 

} 

fciose ( file_symb ) ; 
symbols[max_symb] = 0 ; 

for ( ind = 0 ; ind < 6 ; ind++ ) 
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for ( index = 0 ; index < NUMBER_POINT ; index++ ) { 
param[ind][index] = 0.0 ; 

} 

if ( ( injile = fopen ( file_pen , "rb u ) ) » NULL ) 

strcpy (exLerr.filejDen); 
return -4 ; 

} 

index = 0 ; 
NumSymbols = 0 ; 
symb=-1 ; 

if (adapt) 

repl a make_corr ( param , symbols , symb) ; 
else { 

rep! = make_corr ( param , symbols , -1) ; 

if (rep![0].ndx<0) 
return ( repl[0].weight); 

if (repl[0].ndx=symb) 

Ngood++; 
else 

Ngood = Ngood ; 

} 

fclose ( injile ) ; 
if (NumSymbols=0) return 0; 
else return (Ngood/(float) NumSymbols) ; 



r Calculation of the similarity of all the parameters of all the prototypes and the 
symbol to be recognized 7 

extern float huge *alLpar[l00] ; 

struct reply 
{ 

int ndx; 
float weight; 

}; 

static int comm_count = 0 , abs_count = 0 ; 

int objjunct ( float [100][7] , int , int , float [100] , float [7] , int [10] ) • 

float correLhem ( float [NUMBER_POINT] , float [NUMBER POINT]' float ) ■ 

float correl ( float [NUMBER_POINT] , float [NUMBER POINT] ) ; 

struct reply 'make.corr ( float cur_par[6][NUMBER_pdlNT] , char 'symbols ,int symb) 

FILE *cur_file ; 

int ind_repl , ind_corrct , ind , max_symb , ind_symb , index ; 
struct reply arr_repl[30]; 
int arr.ind[10]; 

float res[100] , nres[7] , old_max_pnt = cur_par[0][0] , com_wight ; 

float old_max_pnt2 , corr[100][7] , tmp_par[6][NUMBER POINT! ; 

charbuf[8]; 

int iterat; 

struct reply rt; 

int i,j; 
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max_symb = strlen ( symbols ) ; 

for ( ind_symb = 0 ; ind_symb < max_symb ; ind_symb++ ) { 
for ( i = 0 ; i < 6 ; i++ ) 
for ( j s 0 ; j < NUMBER.POINT ; j++ ) 
tmp_par[i]0] = alLpar[ind_symb][i*100+j] ; 
if ( tmp_par[0][0] > 0 ) { 
cur_par[0][0] = old_max_pnt ; 
corr[ind_symb][N_PAR-1j = 1.0 • ( 1 - 

min ( fabs ( tmp_par[0][0) - cur_par[0][0] ) / cur_par[0][0] , 1 ) ) ; 

old_max_pnt = cur_par[0][0] ; 
tmp_par[0][0] = 1. ; 
cur_pac[0][0] = 1. ; 

corr[ind_symb)[0] = correLhem ( cur_par[0] , tmp_par{0) , .9 ) ; 
for ( ind = 1 ; ind < N_PAR - 1; ind++ ) { 

corr[ind_symb][ind] = correl ( cur_par[ind] , tmp_parpnd] ) ; 

} 

else 

for ( ind = 1 ; ind < N_PAR - 1; ind++ ) { 
corr[ind_symb][ind] = 0.0 ; 

} 

} 

if (symb<0) 
{ 

index = objjunct ( corr , max_symb , N_PAR , res , nres , arrjnd ) ; 
iterat=20; 

} 

else 

{ 

sprintf ( buf , '%03d.par" , (int) symbois[symb] ) ; 
for ( i = 0 ; i < 6 ; i++ ) 

for ( j = 0 ; j < NUMBER_POINT ; j++ ) 

tmp_par[i][j] = alLparisyrnb][H00+j] ; 
iterat=0; 

while ( (index=objJunct (corr.max.symb.fsLPAR^es.nres, arrjnd))>0 
&& (arUnd[0]i=symb)) 

{ 

if (iterat>19) break; 

for (ind=0, ind_corrct=0; ind<N_PAR-1 ; ind++) 
if (corr[symb][ind]<0.95 * nresfind]) 
{ 

ind_corrct++; 

for (index=0; index < NUMBER_POINT ; index++) 
tmp_par[ind]pndex] = tmp_par [ind][index]\9 
+cur_par[ind][index]M ; 

} 

if (corr[symb][ind]<0.95 * nresflnd]) 
{ 

ind_corrct++; 

tmp_par[0][0] = tmp_par[0][0] • .9 + old_max_pnt * .1 ; 

} 

if (!ind_corrct) { 
iterat = 20 ; 
break; 

) 
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iterat++; 

cur_par[0][0]= old_max_pnt; 

corr[symb][N_PAR-1]=1-fabs(tmp_par[0][0]-cur^par[0][0J)/ cur_par[0][0]; 
qld_max_pnt = cur_par[0][0]; 
old_max_pnt2= tmp_par[0][0]; 
tmpj3ar[0][0J = 1.; 
cur_par[0][0] = 1 .; 

corr[symb][0] = correLhem ( cur_par[0] , tmp_par[0] , .9 ) ; 
for ( ind = 1 ; ind < N_PAR - 1 ; ind++ ) { 
corr[symb][ind] = correl ( cur_par[ind] , tmp parfind] ) ; 

} 

cur_par[0][0] = o!d_max_pnt ; 
tmp_par[0][0] = old_max_pnt2 ; 
} r while 7 
} r else 7 

if ((iterat<20) && (index>0) && (iterat>0)) 

curjile = fopen ( buf , "w+b fl ) ; 

for ( index = 0 ; index < N_PAR -1 ; index++ ) 

fwrite ( tmp_par[index] , sizeof ( float ) , NUMBER.POINT , cur file ) ; 
fclose ( curjile ) ; 
for ( i = 0 ; i < 6 ; i++ ) 

for ( j = 0 ; j < NUMBER_POINT ; j++ ) 
^ alLpar[symb][i*100+j] = tmp_par[i]Q] ; 

index = min ( index ,9); 
arund[index]=-1; 
res[arr_ind[index]l=-1; 
for (i=0;i<=index;w-+) 
{ 

arr_rep!fi].ndx=arUndp]; 
arr_repl[i].weight=-res[arrjnd[i]]; 

} 

return arr_repl ; 

} 



r Calculation of correlation between two vectors */ 

float correl ( fioat first[NUMBER.POINT] , float second[NUMBER_POINT] ) 

float sumxy = 0.0 , sumx = 0.0 , sumy = 0.0 , sumx2 = 0.0 , sumy2 = 00- 
intLd.Ls; ' 
for ( Ls = 0 ; Ls < NUMBER_POINT ; i_s++ ) { 

sumxy += first[i_s] * second[i_s] ; 

sumx += first[i_s] ; 

sumy += secondfLs] ; 

sumx2 += first[Ls] * first[Ls] ; 

sumy2 += second[Ls] * second[Ls] ; 

if ( ( sumx2 - sumx * sumx / NUMBER^POINT ) < 0 II 
( sumy2 - sumy • sumy / NUMBER_POINT ) < 0 ) 
return 0 ; 

if ( ( sumxy = ( sumxy - sumx • sumy / NUMBER_PO!NT ) / 
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sqrt ( sumx2 - sumx * sumx / NUMBER_POlNT ) / 
sqrt ( sumy2 - sumy * sumy / NUMBER_POINT ) ) < .5 ) 
return 0 ; 
return sumxy ; 

} 



r Similarity function for the parameter of pen up/down 7 

float correLhem ( float par1[NUMBER_POIN7] , float par2[NUMBER_POINT] , float border ) 

int index ; 

float result a 0.0 ; 

for ( index = 1 ; index < NUMBER_POINT ; index++ ) 

result +a fabs { pari [index] - par2[index] ) ; 
result /= NUMBER_POINT ; 
result = 1 - result ; . 
if ( result < border ) 

return 0 ; 
return result ; 

} 

r Selection of the list of symbols that are likely to be the symbol to be recognized 7 

int objjunct ( float arr[100j[7] , int n_symb , int n_par , 

^ float res[100] , float nres[7] , int arrindex[30] ) 

int ind_s , ind_p , ind_arr = 0 ; 

float max_res = 0.0 , cur_res , abs_res ■ 0.0 ; 

int result = -1 ; 

for ( ind_s = 0 ; ind_s < n_symb ; ind_s++ ) { 
for ( ind_p = 0 , cur_res * 0.0 ; ind_p < n_par ; ind_p++ ) 

curares += arr[incLs]pnd_p] ; 
res[ind_s] = curares ; 
if ( curares > max_res ) { 

result = ind_s ; 

max_res = curares ; 

} 

} 

abs_res = max_res * .85 ; 
do { 

arrindexpnd_arr++] a result ; 
res[result] = - resfresult] ; 

for ( ind_s = 0 , max_res = 0.0 ; ind_s < n_symb ; ind_s++ ) 
if ( res[ind_s] > max_res ) { 
result = ind_s ; 
max_res = res[ind_s] ; 

} 

} while { max_res > abs_res && ind_arr < 30 ) ; 
for ( ind_p s 0 ; ind_p < n_par ; ind_p++ ) 
for ( ind_s = 0 , nres[ind_p] a -5 ; ind_s < n_symb ; ind_s++ ) 
nres[ind_p] = max ( arr[ind_s][ind_p] , nrespnd_p] ) ; 
return ind_arr ; 

} 
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CLAIMS 



1- Information input apparatus comprising: 

body supported apparatus for sensing voluntary 
bqdy motions and providing an output indication thereof; 

a symbol output interpreter operative to uti- 
lize said output indication for providing symbol outputs; 
and 

a motion output interpreter operative to uti- 
lize said output indication for providing motion control 
outputs . 



2. Information input apparatus according to claim 
1 and wherein said output indication represents features 
of body motion including features which are characteris- 
tic of the individual, 

3. Information input apparatus according to either 
of claims 1 and 2 and also comprising a mode selector 
operative to cause a selected one of the symbol output 
interpreter and the motion output interpreter to func- 
tion. 

4. Information input apparatus according to any of 
the preceding claims and wherein said body supported 
apparatus is a hand held device. 



5 - Information input apparatus according to any of 

the preceding claims and wherein said body supported 
apparatus is a generally pen-shaped device. 

6 * Information input apparatus according to claim 

5 and wherein said generally pen-shaped device is opera- 
tive to provide a visible writing function. 
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7. Information input apparatus according to any of 
the preceding claims and also comprising an object whose 
motion is controlled by said motion control outputs . 

8. Information input apparatus according to claim 
7 s and wherein said object is a graphic object displayed 
on a display. 

9. Information input apparatus according to claim 
7 and wherein said object is a physical object. 

10. Information input apparatus according to any of 
the preceding claims and wherein said symbol outputs 
represent alphanumeric symbols. 

11. Information input apparatus according to any of 
the preceding claims and wherein said symbol outputs 
represent a sensory quality. 

12. Information input apparatus according to claim 
7 and also comprising a computer, having a location input 
and a symbol input, and a display operated by said com- 
puter and wherein said symbol outputs represent informa- 
tion to be displayed on said display and said motion 
outputs are supplied to said location input and are 
employed by the computer to govern the location of said 
information on said display. 

13. Information input apparatus according to claim 
7 or claim 12 and wherein said symbol outputs include 
function commands. 

14 • A method by which a manipulable device provides 

an output indication representing its own angular motion, 
the method comprising: 

recording actual acceleration data from a 
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plurality of accelerometers mounted in the manipulable 
device; 

generating predicted acceleration data on the 
basis of hypothetical angular motion information; 

comparing the predicted acceleration data to 
thf actual acceleration data; 

computing improved hypothetical angular motion 
information; 

while the predicted acceleration data differs 
significantly from the actual acceleration data, repeat- 
ing the generating, comparing and computing steps; and 

providing an output indication of the improved 
hypothetical angular motion information. 



15 • A method according to claim 14 wherein the 

angular motion information includes angular displacement 

information, angular velocity information and angular 
acceleration information. 



16 • A method according to claim 14 or claim 15 . and 
also comprising computing linear motion information from 
the improved hypothetical angular motion information and 
from the actual acceleration data, 

17 • A method according to any of claims 14 - 16 
wherein recording comprises recording from at least four 
accelerometers mounted in the manipulable device, wherein 
the accelerometers each have a center of mass and wherein 
the centers of mass do not lie within a single plane. 

18 • A method according to any of the preceding 
claims 14 - 17 and also comprising receiving the output 
indication of the improved hypothetical angular motion 
information and manipulating an object in accordance 
therewith. 
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19. An acceleroraeter array mounted in a manipulate 
device and comprising: 

at least four accelerometers each having a 
center of mass, wherein the centers of mass do not lie 
within a single plane; and 

a manipulable device motion computer receiving 
input from the accelerometers and generating an output 
signal indicative of the motion of the manipulable de- 
vice. 

20. Apparatus according to claim 19 wherein the 
manipulable device motion computer is operative to per- 
form tiie following steps: 

recording actual acceleration data from the 
accelerometers; 

generating predicted acceleration data on the 
basis of hypothetical angular motion information; 

comparing the predicted acceleration data to 
the actual acceleration data; 

computing improved hypothetical angular motion 
informations- 
while the predicted acceleration data differs 
significantly from the actual acceleration data, repeat- 
ing the generating, comparing and computing steps; and 

providing an output indication of the improved 
hypothetical angular motion information. 

21.. Apparatus according to any of claims 19 - 20 

and also comprising an object manipulator receiving the 
output signal indicative of the motion of the manipulable 
device and manipulating an object in accordance there- 
with. 

22 • An information input method comprising: 

sensing voluntary body motions and providing an 
output indication thereof; 
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utilizing said output indication for providing 
symbol outputs; and 

utilizing said output indication for providing 
motion control outputs. 
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